Risk Sensitive Path Integral Control
نویسندگان
چکیده
Recently path integral methods have been developed for stochastic optimal control for a wide class of models with non-linear dynamics in continuous space-time. Path integral methods find the control that minimizes the expected cost-to-go. In this paper we show that under the same assumptions, path integral methods generalize directly to risk sensitive stochastic optimal control. Here the method minimizes in expectation an exponentially weighted cost-to-go. Depending on the exponential weight, risk seeking or risk averse behaviour is obtained. We demonstrate the approach on risk sensitive stochastic optimal control problems beyond the linear-quadratic case, showing the intricate interaction of multi-modal control with risk sensitivity.
منابع مشابه
A Unifying Framework for Linearly Solvable Control
Recent work has led to the development of an elegant theory of Linearly Solvable Markov Decision Processes (LMDPs) and related Path-Integral Control Problems. Traditionally, LMDPs have been formulated using stochastic policies and a control cost based on the KL divergence. In this paper, we extend this framework to a more general class of divergences: the Rényi divergences. These are a more gen...
متن کاملCost Distribution Shaping: The Relations Between Bode Integral, Entropy, Risk-Sensitivity, and Cost Cumulant Control
The cost function in stochastic optimal control is viewed as a random variable. Then the classical linearquadratic-Gaussian control, entropy control, risk-sensitive control, and cost cumulant control can be viewed as the cost distribution shaping methods. In this paper, we will survey the existing relations between entropy, Bode integral, and risk-sensitive cost function. Furthermore, we will r...
متن کاملNumerical solution of higher index DAEs using their IAE's structure: Trajectory-prescribed path control problem and simple pendulum
In this paper, we solve higher index differential algebraic equations (DAEs) by transforming them into integral algebraic equations (IAEs). We apply collocation methods on continuous piece-wise polynomials space to solve the obtained higher index IAEs. The efficiency of the given method is improved by using a recursive formula for computing the integral part. Finally, we apply the obtained algo...
متن کاملA Unified Theory of Linearly Solvable Optimal Control
We present a unified theory of Linearly Solvable Optimal Control, that is, a class of optimal control problems whose solution reduces to solving a linear equation (for finite state spaces) or a linear integral equation (for continuous state spaces). The framework presented includes all previous work on linearly solvable optimal control as special cases. It includes both standard control problem...
متن کاملAn Iterative Path Integral Stochastic Optimal Control Approach for Learning Robotic Tasks
Recent work on path integral stochastic optimal control theory Theodorou et al. (2010a); Theodorou (2011) has shown promising results in planning and control of nonlinear systems in high dimensional state spaces. The path integral control framework relies on the transformation of the nonlinear Hamilton Jacobi Bellman (HJB) partial differential equation (PDE) into a linear PDE and the approximat...
متن کامل